The existing literature and/or patents on noise reducing techniques are abundant. Image de-noising techniques may be classified as spatial or temporal ones or a combination of them. Spatial techniques relate generally to some coring techniques applied to high frequency part of a considered image. Temporal de-noising techniques relate to temporal coring techniques applied mainly in detected or estimated still parts of a picture. Image de-noising techniques may be classified as spatial or temporal ones. A series combination of the spatial and temporal techniques, easier to do than a true 3D processing, is possible and may be of advantage. In the following, the general trend of the subject will be reviewed and the specific spatial or temporal exiting noise reducers will be considered in some details.
The spatial noise reducing techniques may be applied to either still pictures or to a sequence of images. In general, the spatial noise reducing techniques may be divided further into three categories.
In the first category, the spatial nonlinear filters are based on local order statistics. These techniques may be found, for example, in A. R. Weeks Jr., “Fundamentals of Electronic Image Processing”, SPIE Optical Engineering Press, Bellingham, Wash., 1996 or I. Pitas, and A. N. Venetsapoulos, “Nonlinear Digital Filters: Principles and Applications”, Kluwer Academic Publishers, Boston, 1990. Using a local window around a considered pixel, these filters are working on this set of pixels ordered now from their minimum to their maximum values. The median filter, the min/max filter, the alpha-trimmed mean filter, and their respective variants may be classified in this category. These filters work well for removing impulse like salt-and-pepper noise. For the small amplitude noise these filters can blur some details or small edges.
In the second category, the coring techniques are applied in another domain different from the original image spatial domain. The chosen domain depends partly on the noise nature. The U.S. Pat. No. 4,163,258 uses the Walsh-Hadamard transform domain; meanwhile, the U.S. Pat. No. 4,523,230 suggests some sub-band decomposition. Finally, the homomorphism filter, working in logarithmic domain, is the classical one for removing multiplicative noise and image shading from an image.
In the third category, the filters are locally adaptive and the noise removing capacity is varying from homogenous regions to edge regions.
The well-known filter in this category is the minimum mean square error (MMSE) filter proposed originally by J. S. Lee in “Digital image enhancement and noise filtering by use of local statistics”, IEEE Trans. on PAMI-2, March 1980, pp. 165-168. The filtered pixel output is additively composed of local mean value and a pondered difference of the noisy pixel and the local mean intensity values. The optimum weight, which corresponds to a kind of coring technique, may be determined for additive noise by the local variance ratio of the true clean image and the noisy one. The minimum mean square error filter removes noise well for homogenous regions and reserves the image edges. However, the noise essentially remains in edge or near edge regions. Moreover, the optimum weight is changing for other types of noise.
A relationship of Lee's filter and recent Anisotropic Diffusion techniques may be shown by Y. Yu and S. T. Acton in “Speckle Reducing Anisotropic Diffusion”, IEEE Trans. on Image Processing, vol. 11, November 2002, pp. 1260-1270.
In P. Chan and J. S. Lim, “One dimensional processing for adaptive image restoration”, IEEE Trans. on ASSP-33, February 1985, pp. 117-126, there is presented a method for noise reducing in edge regions. The authors have proposed the use, in series, of four (4) one-dimensional minimum mean square error filters respectively along 0°, 45°, 90° and 135° directions. The obtained results are impressive for large variance noise. For small noise, the filter can blur however some image edges. Moreover, the noise variance output at each filter stage is to be costly estimated.
For the same purpose, in J. S. Lee, “Digital image smoothing and the Sigma filter”, Computer Vision, Graphics, and Image Processing-24, 1983, pp. 255-269, the author has proposed a Sigma filter. For noise removing, the filter calculates, in a local window of 5×5 dimensions, the mean value of similar pixel intensities to that of the central considered pixel. For small noise, the Sigma filter works well, except small image details and some pixels with sharp spot noise. For the latter, J. S. Lee has suggested also, in a heuristic manner, the use of immediate neighbor average at the expense of some eventually blurred picture edges.
U.S. Pat. No. 4,573,070 discloses a Sigma filter for a 3×3 window. The author has combined, in a single configuration, the Sigma filter, an order statistic filter and a strong impulse noise reduction filter.
In U.S. Pat. No. 6,633,683, a Shape adaptive Windowing combined both minimum mean square error and Sigma Filter techniques, is disclosed. However, introduced banding artifact effect in slowly varying regions and generic minimum mean square error structure for some usual types of noise are not considered.
The temporal filter is generally applied for a sequence of images in which the noise component is supposed to be non-correlated between two or many successive images. The temporal filtering techniques are based essentially on motion detection (MD) or motion compensation (MC). The filter structure may be IIR (infinite impulse response) or FIR (finite impulse response) filter with frame delay elements. In general, the temporal techniques perform better than spatial ones. The system cost is due essentially to the frame memory and the motion estimation. The temporal de-noising techniques may be found, for example, in U.S. Pat. Nos. 5,161,018, 5,191,419, 5,260,775, 5,404,179, 5,442,407, 6,061,100 and in G. Wischerman, “The Digital Wetgate: A Third-Generation Noise Reducer”, SMPTE Journal, February 1996, pp. 95-100.
From a theoretical standpoint, a class of these noise filtering techniques based on well established M C Kalman filtering is proposed for spatio-temporal domain in Kim and Woods, “Spatio-temporal adaptive 3-D Kalman filter for Video”, IEEE Transactions on Image Processing, Vol. 6, No. 3, March 1997. However, 3D Kalman filter is not convenient for high speed implementation or abrupt scene change. Katsaggelos and al. in “Adaptive Image Sequence Noise Filtering Methods”, SPIE Vol. 1606 Visual Communication and Image Processing 1991, pp 716-727, have proposed two approaches for non stationary filtering of image sequences: a separable adaptive recursive motion compensated filter composed of three coupled 1-D estimators and a temporal non-linear filtering approach without motion estimation. M. K. Ozkan et al. in “Adaptive Motion Compensated Filtering of Noisy Image Sequences”, IEEE Trans. on Circuit and Systems for Video Technology, Vol. 3, No. 4, Aug. 1993, pp 277-290 have suggested the use of adaptive weighted averaging filter for claiming to overcome presence of edge, inaccurate motion estimation and scene change. Boo and Bose in “A motion-compensated spatio-temporal filter for image sequences with signal dependent noise”, IEEE Trans. on Circuit and Systems for Video Technology, Vol. 8, No. 3, June 1998, pp 287-298 have proposed a MC spatio-temporal filter using groups of frame and LMMSE in a transform domain.
The most interesting for the present invention is the second approach of Katsaggelos and al.: a temporal non-linear filtering approach without explicit motion detection or estimation. However, their approach is costly in implying five frame memories and an inversion of matrix.
U.S. Patent Application No. 2001/0019633 discloses using a kurtosis of the noise as a metric for estimating the type of noise and applied either median filter or spatio-temporal filter in function of noise discrimination.